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Abstract 


One commonly-used strategy used in attempts to decrease racial disproportionality in school 
discipline across the country is sharing data with school administrators that discipline disparities 
are a problem in their schools, with the assumption that it will increase attention to equity and 
improve outcomes. The purpose of this study was to assess the effects of providing monthly 
disciplinary equity reports to school administrators in 35 schools on levels of (a) disciplinary 
equity report viewing, (b) disciplinary equity, and (c) inclusion of disciplinary equity into school 
improvement plan goals. We used a small, double-blind randomized controlled trial, in which 
half of the schools were randomly assigned to receive either monthly disciplinary equity reports 
or monthly general discipline reports. Results showed that schools receiving the equity reports 
had significantly increased rates of viewing equity reports, but no meaningful change in 
disciplinary equity or equity goal setting. 

Keywords: Racial Equity, Performance Feedback, Data, School 


Discipline 
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Awareness is Not Enough: A Double-blind Randomized Controlled Trial of the Effects of 
Providing Discipline Disproportionality Data Reports to School Administrators 

The disproportionate use of office discipline referrals (ODRs) for Black students is a 
longstanding concern in education (U.S. Government Accountability Office, 2018). Although 
there is evidence for effectiveness of some systems-level interventions, such as positive 
behavioral interventions and supports (McIntosh, Gion, & Bastable, 2018), implementing these 
interventions with fidelity requires administrator commitment and substantial resources; thus, 
district-wide implementation can be challenging. Accordingly, there is a tendency to seek 
efficient interventions that can be implemented quickly. 

To motivate school administrators to take action, districts or state teams in multiple states 
have adopted the intervention of sending school administrators regular (e.g., monthly) reports 
showing the extent to which their schools have racial disparities in exclusionary discipline 
(Santiago-Rosario, 2019). Although minimal in terms of effort, sending equity reports has 
intuitive appeal as a way to draw attention to and signal the importance of addressing the 
problem (Kluger & DeNisi, 1996). Further, there is evidence that school teams that use data 
more often for decision making also have lower racial discipline disproportionality (Tobin & 
Vincent, 2011). Even so, at the school level, simply making equity reports available to school 
administrators on demand does not necessarily increase their viewing of discipline data 
(McIntosh, Eliason, Horner, & May, 2014). Moreover, sharing reports showing racial inequities 
could backfire, increasing resistance to equity efforts (Hetey & Eberhardt, 2018; Kluger & 
DeNisi, 1996). 


Purpose of the Study 
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The study’s purpose was to provide a rigorous experimental test of providing disciplinary 
equity data reports to school administrators. We assessed the effects of providing monthly 
disciplinary equity reports, compared to a control of monthly discipline summary reports, to 
school administrators on (a) viewing disciplinary equity reports, (b) equity in school discipline, 
and (c) identifying disciplinary equity as a school improvement goal. 

Method 
Settings and Participants 

A total of 35 public K-12 schools and their school administrators within three school 
districts in the U.S. Pacific Northwest participated in the study. The 26 elementary, 5 middle, and 
4 high schools used the School Wide Information System (SWIS; May et al., 2018) to record and 
review discipline data. The average Black-White ODR risk ratio was 2.2 (ranging from 0 to 6.3), 
meaning that on average, Black students were more than twice as likely to receive one or more 
ODRs than White students. 

Measures 

Viewing of school discipline reports. Through SWIS, we extracted counts of discipline 
data reports (1.e., sets of graphs and tables) generated by school users per month. We 
standardized the data by using the metric of reports generated per 20 school days, roughly a 
month of school. 

Equity in school discipline. We assessed equity in school discipline using the Black- 
White ODR rate difference, calculated by subtracting the ODRs per White student per day per 
month from the ODRs per Black student per day per month. This metric of disciplinary equity 
accounted for enrollment of each student group and varying school days in each month. A rate 


difference of zero indicates perfect equity in school discipline, and positive values indicate 
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higher ODR rates for Black students. The rate difference was selected because, unlike risk ratios, 
it has a symmetrical distribution based on a standard scale (change in rates) and thus is more 
appropriate for computation of difference scores without transformation and can be calculated 
when one group receives zero ODRs (Girvan, McIntosh, & Smolkowski, 2019). 

School improvement plan goal setting. We obtained publicly available, annual school 
improvement plans for each of the participating schools from 2016-17 and 2017-18, which 
described their formal academic and behavior goals. We coded the plans for whether they 
included disciplinary equity as a goal (intercoder agreement = 88%). 

Intervention 

The intervention was a monthly email to school administrators containing one of the 
continuously-available SWIS school discipline data reports (see http://www.pbisapps.org): either 
the (a) School Summary Report (control condition), which includes figures and tables showing 
general patterns of ODRs; or (b) School Ethnicity Report (intervention condition), which 
includes figures and tables showing the extent of racial discipline disproportionality. No other 
communication or support was provided. 

Procedure 

Once recruited, schools were matched into pairs within districts based on their 
disciplinary equity (specifically the Black-White ODR risk ratio) and randomly assigned to 
condition in January of Year 2. School administrators were emailed the reports on the first day of 
each month for the remaining five months of the school year (one school was lost from the 
sample due to not receiving its report in time). Figure 1 shows the participant flow diagram. The 
school and district administrators, lead author (principal investigator), second author 


(methodologist), and fifth and sixth authors were all blinded to condition until all analytic 
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decisions were completed. Only the third and fourth authors, who generated and sent the reports, 
were aware of condition during the study, and they did not participate in analytic decisions. 
Data Analysis 

Report viewing and disciplinary equity. Analyses assessed the effects of receiving the 
emailed equity reports on (a) rates of viewing disciplinary equity reports and (b) actual discipline 
disparities. For both DVs, we subtracted Year 1 data for each month from Year 2 data for the 
same month to produce year-on-year monthly change over time. Thus, the analyses directly 
addressed the hypothesis that the schools that received equity reports compared to general 
reports would experience an immediate increase in their viewing of reports and disciplinary 
equity above levels from the same time in the previous year. The supplemental appendix 
provides a detailed description of data analysis. 

School improvement plan goal setting. We evaluated each school’s school 
improvement plan for 2016-17 and 2017-18 to assess the extent to which the intervention 
condition group had an increased proportion of schools including goals to increase racial equity 
in school discipline. We conducted a chi-square test of proportions across groups to assess 
statistical significance. 

Results 
Report Viewing and Disciplinary Equity 

Figure 2 presents the year-on-year mean differences for equity report viewing and 
disciplinary equity by condition, with 95% confidence intervals, pre and post intervention. 
Results showed a statistically significant increase in equity report views (p = .023, Hedges’ g = 
0.65) upon intervention for the intervention schools but no decrease in Black-White ODR 


differences, while control schools saw a decrease (p = .155, Hedges’ g = 0.09). The supplemental 
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appendix provides detailed results. 
School Improvement Plan Goal Setting 

Prior to intervention, no schools in either condition included school improvement plan 
goals referring to increasing racial equity in school discipline. After intervention, no control 
schools and one intervention school reported a school improvement plan goal of increasing 
disciplinary equity. A 7° analysis of post-test proportions showed that the one-school increase in 
equity goal setting was not statistically significant (77 = 1.09, p > .05). 

Discussion 

Given the need to address disciplinary equity, district and state teams are increasingly 
engaging in the practice of sending school administrators reports of their disciplinary equity. Our 
study examined the effects of directly sending monthly disciplinary equity reports to school 
principals on (a) accessing data, (b) disciplinary equity, and (c) equity goal setting. The double- 
blind randomized controlled trial design with an attention control provided an opportunity to 
assess the effects of this common intervention with less susceptibility to reactivity or social 
desirability bias. 

Regarding data report viewing, although they had already been available to school 
administrators, sending equity reports directly to them significantly increased how much they 
accessed and viewed the reports. Although the increase in equity report viewing was 
encouraging, the provision of reports did not lead to any meaningful changes in disciplinary 
equity or school improvement plan goal setting. As such, we view this intervention as ineffective 
in improving equity on its own. This study adds to previous research in performance feedback 
and equity data that simply sharing data showing inequities could have iatrogenic effects, 


especially if recipients perceive that they do not have sufficient control of the context to achieve 
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equitable disciplinary outcomes (Bastable & McIntosh, 2020; Hetey & Eberhardt, 2018; Kluger 
& DeNisi, 1996). 
Limitations 

Although tightly controlled and rigorous, this study has limitations worth considering. 
First, due to software limitations, we could not assess which users were accessing reports. As a 
result, it is unclear whether the increase in viewing came from school administrators or other 
school or district personnel. Second, the study was small and may have been under-powered to 
detect changes in ODRs. 
Implications 

Our findings indicate that sending equity reports directly to school administrators 
increases attention to data, although such effects do not appear to change equity in school 
discipline or goal setting. This study suggests that when provided evidence of disproportionality, 
school administrators are unlikely to take important actions, like setting equity goals in their 
school improvement plans. Yet it is noteworthy that providing reports increased views, which 
has been an elusive outcome (McIntosh et al., 2014). More research is needed to determine how 


this common intervention might be supplemented to increase goal setting or disciplinary equity. 
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Figure | 


Study Participant Flow Diagram 


Assessed for eligibility (n = 65) 


Excluded (n = 29) 
*Declined to participate (n = 29) 


Randomized (n = 36) 


Assigned to intervention (n = 18) 
Assigned to treatment in time (n = 17) 
*Received intervention (n = 17) 


Analyzed (n= 17) ; 
«Included in goal analyses (n = 17) Analysis 
«Included in pre-post SWIS analyses (n = 15) 


Note. The figure depicts the number of schools considered for the study and their participation in the two conditions 
at the assignment and analysis phases. 


Assigned to control (n = 18) 
Assigned to treatmentin time (n = 18) 
«Received control condition (n = 18) 


Assignment 


Analyzed (n= 18) 
«Included in goal analyses (n = 18) 
«Included in pre-post SWIS analyses (n = 15) 
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Figure 2 


Mean Year-On- Year Differences in Equity Report Views per 20 School Days (top) and Black- 
White ODR Rate Differences per Day (bottom) 
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Note. Each graph presents the average year-on-year differences over the five months before intervention (Pre) and 
the five months after intervention (Post) separately for each condition. The error bars represent 95% confidence 
bounds. 
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Awareness is Not Enough: A Double-blind Randomized 
Controlled Trial of the Effects of Providing Discipline 
Disproportionality Data Reports to School Administrators 
Supplemental Appendix 
In this double-blind randomized controlled trial, we randomly assigned schools to one of two 
conditions and compared conditions on the number of discipline reports viewed and equity 
metrics for two years. The intervention began in January of the second year. We examined 
schools’ year-on-year differences for each measure to control for month-to-month fluctuations in 
data. This supplemental appendix provides additional details about the statistical analysis and 
results that were not presented in the brief report. 
Data Analysis: Report Viewing and Disciplinary Equity 
The analyses were intended to address the hypothesis that schools that received equity reports 
halfway through the second year would increase their views of the equity reports and other SWIS 
reports. The reports delivered to the school as part of the study, such as the equity reports 
provided to intervention schools, were not included in the data. Due to differences in school 
days, SWIS data varies substantially from month to month. We accounted for month-to-month 
variability in two ways. First, we analyzed report data per 20 school days, roughly a month of 
school, and analyzed average ODR data per day for each month to normalize the data for varying 
school days in each month. Second, we subtracted the Year 1 data from the Year 2 data to 
produce year-on-year monthly change over time. Specifically, for each dependent variable (DV), 
we subtracted the September data collected in Year 1 from the September data collected in Year 
2 and repeated this process for each month during the school year. This procedure produced 


more stable data that can be interpreted as changes from Year 1 to Year 2 for each month. The 
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analysis of these data addressed the hypothesis that midway through the school year, for 
example, the schools that received equity reports increased views of these reports during the 
latter half of the year over those viewed in the previous year. 

For any test of condition effects, the correct interpretation of model parameters and associated p 
values depends on the validity of the underlying model (Wasserstein & Lazar, 2016). “The P- 
value simply indicates the degree to which the data conform to the pattern predicted by the test 
hypothesis and all the other assumptions used in the test (the underlying statistical model)” 
(Greenland et al., 2016, p. 4). Because delivery of reports may have (a) changed the level of the 
DVs or (b) increased the DVs over time in a linear fashion across the second half of the school 
year, the analysis fit the data for each DV to two different models: Model A, a constant change 
model, and Model B, a linear change model. Both models allowed for linear growth over time 
for control schools and constrained the two conditions equal until the beginning of the 


intervention period. Figure Al shows the two competing models. 


We then selected the most likely model given the pair and the underlying data in order to use the 
most valid model, as per Greenland et al. (2016), before interpreting condition effects. To select 
the most valid and generalizable model for each DV, we compared the models using a small- 
sample bias correction variant of the Akaike Information Criterion (AIC; Burnham & Anderson, 
2002), an index called the AICc. Because the AICc is a relative measure of fit, we calculated the 
A AICc, the difference between the AICc value from each model and the minimum AICc value 
among the pair of models (Burnham & Anderson, 2002). The model with the smallest AICc 
value has a A AICc of zero and the greatest support from the data. Plausible alternative models 
tend to have A AlCc values less than about 2 to 4, and values of 6 to 10 or more indicate limited 


support over the best-fitting model (Burnham & Anderson, 2002). 
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We also reported Akaike weights, which express the relative likelihood of a model given the data 
and the set of models under consideration (Burnham & Anderson, 2002). The weights, also 
called model probabilities, describe the strength of evidence for a model. Specifically, they 
quantify the strength of evidence for each hypothesis, represented by a statistical model, given 
the data and all other hypotheses (models) tested (Anderson, 2008; Burnham, Anderson, & 
Huyvaert, 2011). The weights, denoted w for each model, sum to | across the two models for 
each DV. The weight, w, can be interpreted as the probability that the same model would be 
selected with a “replicate data set from the same system” (Burnham et al., 2011, p. 30) and, in 
the general case with several models, “allow statements such as ‘the probability of H4 is 0.78, 
while the probability of H2 is 0.015’” (Burnham et al., 2011, p. 26). If both models have similar 
weights, one can inspect parameters from both models (1.e., multimodel inference; Burnham & 
Anderson, 2002). Once we selected a model, we interpreted condition effects. 

Condition effects were similarly examined with model probabilities. To examine condition 
differences, we compared models for two hypotheses, one that hypothesized an intervention 
effect (Ha) and one that did not (Ho) and reported the Akaike weight, w, for the model with the 
condition effect (Ha). For example, if w = .75, it suggests the probability of Ha is .75 while the 
probability of Ho is .25. Incidentally, w = .75 is often roughly equivalent to p = .05, which better 
communicates the tenuous nature of “just-significant” results (p ~ .05) because the model for Ha 
is only three times as likely as the model for Ho given the two models and data. 

We fit the data to our models with SAS PROC MIXED (SAS Institute, 2016) using the full 
information maximum likelihood estimation (FIML). Restricted (residual) maximum likelihood 
(REML), generally recommended for multilevel models (Snijders & Bosker, 2012), does not 


include fixed effects in the log-likelihood function, which makes it inappropriate for 
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comparisons between models that differ on fixed effects. In analyses with missing data, 
maximum likelihood estimation (a) minimizes the likelihood of biased parameter estimates, (b) 
uses all available information to produce efficient estimates with minimum sampling variability, 
and (c) produces accurate estimates of precision, such as standard errors and p values (Allison, 
2009; Graham, 2009). All tests of condition relied on 28 degrees of freedom. 

Results 
Table Al reports mean ODR rates for Black and White students by treatment condition and 
month. Table A2 reports means and standard deviations for the year-on-year differences for each 
dependent variable by month of the school year and intervention condition. Figure A2 presents 
the year-on-year mean differences for each dependent variable graphically by month and 
condition with 95% confidence intervals. For this analysis, we included the 15 schools in each 
condition that contributed data for both years. 
The statistical models constrained the trajectories over time to be identical across conditions for 
the first five months and then allowed the intervention condition to differ in the last five months. 
We characterized potential differences in two possible ways: a constant change or linear change, 
both beginning in Month 6 (see Figure Al). The top two rows of Table A3 summarize the ability 
of the two models to approximate the data at hand for each DV with the A AICc and Akaike 
weights, w. For equity report views, the constant-change model (w = .70) was just more than 
twice as likely as the linear change model. For Black-White ODR rate differences, the linear 
model (w = .71) was the more likely of the two models given the data. Hence, we interpreted the 
constant model equity report views and the linear model for the Black-White ODR rate 
difference but with some caution as the model selection statistics were somewhat equivocal. 


The most likely models indicate an increase in equity report views with the constant-change 
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model, Hedges’s g = 0.65, 95% CI [0.10, 1.20], t23 = 2.41, p = .023, w = .86, after the 
introduction of intervention halfway through the second school year. Here w represents the 
difference between a model with the condition difference and a model without and implies a 6:1 
ratio of support for the model with condition differences. The Condition at Intercept parameter 
estimates the difference between conditions within the first five months of the school year and 
favors the control condition (g = —0.17). We found no evidence of condition differences on 
Black-White ODR rate differences with the more-likely linear-change model, g = 0.09 [-0.04, 
0.21], 3 = 1.46, p = .155, w=.51. The Condition at Intercept parameter suggests the difference 
between conditions favors the control condition (g = —0.16). 

The effect sizes require some explanation. For the constant-change models, the effect sizes in 
Table A3 straightforwardly describe the size of the constant change. In the linear-change models, 
in contrast, the effect size describes the size of the per-unit-time change. The Hedges’s g for 
equity report views was 0.65 for the more-likely constant-change model, but 0.16 per time point 
in the linear-change model, which appears to be inconsistent. The effect size for the linear- 
change model, however, was 0.16 in Month 6, 0.31 in Month 7, and so on up to 0.78 in Month 
10. The average of the monthly effect sizes was 0.47, which is closer to the g of 0.65 described 
by the constant model. The intervention, therefore, appeared to increase equity report views but 


not affect Black-White ODR rate differences in the participating schools. 
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Table Al 


Raw Mean Rates of Black and White ODR Rates by Month and Treatment Condition 


ODR Rates for Black Students ODR Rates for White Students 

Month in 
Control Intervention Control Intervention 

School 
Year Year1 Year2 Year1 Year2 Year1 Year2 Year1 Year2 Days 
1 3.23 4.50 5.19 5.30 3.81 5.24 3.50 3.94 19.3 
2 5.54 11.21 11.19 9.41 5.76 5.48 7.04 7.90 20.5 
3 8.72 9.29 7.96 14.22 4.98 6.34 5.26 8.36 17.2 
4 10.60 7.50 6.92 4.75 4.49 3.47 4.63 4.86 12.8 
5 9.55 9.66 10.72 17.16 5.53 7.31 5.28 11.85 19.0 
6 8.50 9.56 11.53 14.48 5.87 7.15 5.80 10.65 17.5 
7 16.66 12.56 18.15 19.98 8.43 8.67 6.85 14.27 22.0 
8 12.34 9.37 15.57 16.78 6.13 6.11 4.64 9.03 15.5 
9 15.29 11.45 20.74 29.76 8.34 10.19 6.71 10.60 21.3 
10 4.98 7.42 9.73 1431 2.27 4.76 3.20 6.13 13.5 


Note. The table reports the mean ODR rates per 100 students per month across schools. Intervention began in Month 
6 of Year 2 (shaded). The Days column reports the average number of school days per month across districts and 
years, which was similar, within | or 2 days, across districts and years for most months and never varied by more 
than 5 days (Month 1; mean difference of 2.8 days across months). Using the number of days specific to each 
district and year, the Year-on-year differences in Black-White ODR rate differences per day, used as the dependent 


EFFECTS OF PROVIDING DISCIPLINARY EQUITY REPORTS 


variable in the analysis, equals [(Rateptacky2 — Ratewnite-y2) — (Ratepiack.y1 — Ratewnite.y1)] / Days. 


20 
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Table A2 


Year-on-Year Differences in Equity Report Views and Black-White ODR Rate Differences by 
Month and Treatment Condition 


Year-on-Year Differences in Equity Report | Year-on-Year Differences in Black-White 


Views per 20 School Days ODR Rate Differences per Day 
Month in Control Intervention Control Intervention 
School 
Year Mean SD Mean SD Mean SD Mean SD 
Before Intervention (Pre) 
1 -0.35 1.68 -0.60 1.78 0.00 0.28 -0.04 0.60 
2 0.60 2.61 -0.56 1.88 0.30 0.42 -0.12 0.76 
3 1.08 4.04 -0.06 0.82 -0.07 0.77 0.16 0.72 
4 0.46 1.34 0.33 0.93 -0.09 1.06 -0.18 0.76 
2) -0.51 2.86 -0.01 0.79 -0.10 0.68 -0.03 0.80 
After Intervention (Post) 
6 -0.27 1.72 -0.01 0.61 -0.01 0.53 -0.08 0.91 
2 -0.48 1.02 0.43 1.39 -0.18 0.51 -0.24 0.99 
8 -0.08 0.75 Lee 1.12 -0.17 0.81 -0.17 1.51 
9 0.18 0.80 0.30 1.33 -0.28 0.55 0.22 1.22 
10 -0.20 1.00 O19 0.52 -0.04 0.61 0.04 0.86 


Note. The table reports the mean number of equity report views per 20 school days and the Black-White ODR rate 
differences per day after subtracting each month in Year | from the same month in Year 2. See the note to Table Al 
for the calculation of Black-White ODR rate differences per day. 
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Table A3 


Results of Analyses to Select the Best Approximating Longitudinal Growth Model for Each 
Dependent Variable and Test Condition Differences in the Second Half of the School Year 


Equity Report Views per 20 Black-White ODR Rate 
School Days Differences per Day 
Constant Linear Constant Linear 
Model A AICc 0.0 1.7 1.8 0.0 
Selection 
w .70 30 29 71 
Fixed Effects Intercept 0.19 0.16 0.03 0.10 
(.28) (.29) (.14) (.14) 
Condition at -0.29 -0.17 -0.02 -0.08 
Intercept (.28) (.27) (.15) (.15) 
Time -0.03 -0.02 -0.02 -0.03 
(.04) (.04) (.02) (.02) 
Condition by 0.84 0.20 0.09 0.07 
Time (.35) (.10) (.16) (.04) 
Variances School 0.08 0.08 0.08 0.08 
(.10) (.10) (.03) (.03) 
Residual 2.80 2.82 0.56 0.56 
(.24) (.24) (.05) (.05) 
Hedges's g' Condition by 0.65 0.16 0.12 0.09 
and 95% CI Time [0.10, 1.20] [0.00, 0.32] [-0.31, 0.55] [-0.04, 0.21] 
pate Sb RE 9 0535 5797 1547 


Time 


Note. The A AICc is the difference between the small-sample corrected Akaike Information Criterion (AICc) for 
each model and the AICc for the best model among each pair of models for the same dependent variable. Akaike 
weights (model probabilities), w, express the relative probability that the data support the model under consideration 
given the data and the pair of models; for each DV, w values sum to 1.0 across each pair. Fixed effects and variances 
shown with standard errors in parentheses. Tests of condition, which produced p values, used 28 degrees of freedom. 


* Hedges’s g for constant models represent the overall intervention effect size whereas the g for linear models 
represent the effect per unit change (1.e., change per month). 
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Figure Al 


Potential Models of Change 


Analytic Models for Year 2 — Year 1 Differences 
Ethnicity Report Study 
A. Constant Change Y=l+Tx+L+TxxCt+e 


Assumes identical trajectories 
in both conditions until the 
intervention period, when 
intervention schools change by 
a constant level. 


B. Linear Change Y=1+Tx+L+TxxLt+e 
Assumes identical trajectories 
in both conditions until the 
intervention period, when 
intervention schools change in 
a linear fashion. 


Model equation terms 


Y = dependent variable Legend 

| = intercept - 

Tx = treatment condition ——— Ethnicity Reports 
(reports =1,controlh=O) 9" Control 

C = constant 

L = linear effect of time Graphs assume higher 

e = error scores are better 


Note. The two potential models of change for each dependent variable demonstrate the expected pattern of data 
before and after the delivery of school summary (control) or ethnicity (intervention) reports. Model A describes a 
constant change in the level of the dependent variables immediately after the introduction of the reports. Model B 


describes a linear change that increases regularly over time during the second half of the school year after the 
delivery of reports. 
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Figure A2 


Monthly Year-On-Year Differences in Equity Report Views per 20 School Days (top) and 


Monthly Average Black-White ODR Rate Differences per Day (bottom) 
4 —_____ Baseline Period Intervention Period 


Year-on-Year Mean Differences for 
Equity Report Views per 20 Days 


—#— Intervention Schools 


. == Control Schools 
-2 T T T T * T T T T T 
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Month of the School Year 
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-0.4 


—#— Intervention Schools 
== Control Schools 
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Year-on-Year Meand Differences for 
Black-White Major ODR Rate Diff per Day 
[o) 

oO 


1 2 3 4 5 6 7 8 9 10 
Month of the School Year 


Note. The top figure depicts the mean number of equity report views per 20 school days by month. The bottom 
figure shows the Black-White ODR rate differences per day averaged over each month. Each graph reports year-on- 
year differences to control for month-to-month fluctuations in data (e.g., Month 1 values from Year 2 minus Month 
1 values from Yearl, etc.). The dark lines show mean levels and the lighter, outer lines represent 95% confidence 
bounds. The vertical line in the center indicates the beginning of the intervention period, where control schools 
received school summary reports and intervention schools received equity reports. 


